Select Columns

val Data = Seq(("James", "Sales", 3000),("Michael", "Sales", 4600),("Robert", "Sales", 100),("Maria", "Finance", 3000),("James", "Sales", 3000),("Scott", "Finance", 3300),("Jen", "Finance", 3900),("Jeff", "Marketing", 3000),("Kumar", "Marketing", 2000),("Saif", "Sales", 4100))

val df = Data.toDF("employee_name", "department", "salary")

Select particular columns form DataFrame
df.select($"employee_name",$"department",$"salary").show


Select particular number of records from  DataFrame(say  4 records)
df.select($"employee_name",$"salary").show(4)


Select all columns form DataFrame
df.select("*").show(4)


To access the column form dataframe
There are multiple ways to access the dataframe.

import org.apache.spark.sql.functions.col
df.select(col("employee_name")).show

import org.apache.spark.sql.functions.column
df.select(column("employee_name")).show



import spark.implicits.symbolToColumn
df.select('employee_name).show



No comments:

Post a Comment